Evaluating and Comparing Deep Models

To help evaluate Deep Learning segmentation quality, this software release features a model evaluation tool that lets you compare different models with selected metrics, such as binary cross entropy, mean absolute error, and Poisson for regression, as well as Jacard similarity, accuracy, and Dice for binary segmentation.

Choose Artificial Intelligence > Deep Learning Model Evaluation Tool on the menu bar to the open the Model Evaluation dialog, shown below.

Model Evaluation dialog

How to Evaluate and Compare Models

1. Load the training set(s) that will provide the input(s) and output(s) for the evaluation, as well as any mask or masks that you intend to apply..
2. Choose Artificial Intelligence > Deep Learning Model Evaluation Tool on the menu bar.

   The Model Evaluation dialog appears.

3. Choose the required input in the Input drop-down menu.
4. If required, select the number of slices used to train the model(s) you want to evaluate. Enter '1' if you trained the model(s) in 2D and '3' or more if you trained the model(s) in 3D.
5. If you selected a multi-slice model, choose a reference slice and spacing.
6. Choose the required output in the Output drop-down menu.

   All saved models that match the input and output criteria appear in the dialog. If required, you can filter the list by entering text in the Filter edit box.

7. Choose a mask in the Mask drop-down menu, optional.
8. If your model(s) was trained with multiple inputs, click the '+' button to add a new training dataset and then choose the required input int he Input drop-own menu.

   All saved models that match the input and output criteria appear in the dialog. If required, you can filter the list by entering text in the Filter edit box.

9. Select the model(s) that you want to evaluate.
10. Choose the required metrics in the Metrics drop-down menu.
11. Click the Compute button.